1. Field of the Invention
The present invention relates to an image processing apparatus for applying a predetermined signal processing to the image taken in the form of electrical signals, and an image pickup apparatus utilizing such image processing apparatus, and more particularly an image processing apparatus having a function of enlarging a taken image, and an image pickup apparatus employing such the image processing apparatus.
2. Related Background Art
With the recent development of the digital image processing technology for digitizing and processing the electrical image information obtained with an image pickup device, the function called xe2x80x9celectronic zoomingxe2x80x9d or xe2x80x9celectronic teleconverterxe2x80x9d is widely adopted in the video cameras or the like for the consumer use.
As an example, FIG. 1 shows a video camera 700 provided with the electronic teleconverter function.
In this video camera 700, the electronic teleconverter function is turned on and off by shifting an electronic teleconverter switch 710 to a terminal A or B.
At first there will be explained the operation of the video camera 700, which is performed when the electronic teleconverter function is turned off by placing the electronic teleconverter switch 710 at the terminal A.
In this state, a switch 706 is switched to a terminal A under the control of a control circuit 714.
The image of an unrepresented object is transmitted through unrepresented plural optical lenses, a zoom lens 701 provided coaxially therewith and an optical low-pass filter 702, and is focused on the image-pickup surface of a CCD (charge-coupled device) 703, thereby being converted (hereinafter called photoelectric conversion) into an electrical image signal.
The image signal obtained in the CCD 703 is supplied, through an amplifier circuit 704, to an analog/digital (A/D) converter 705.
The A/D converter 705 digitizes the image signal, supplied through the amplifier circuit 704, into digital image data, which are supplied to a terminal A of the switch 706.
Since the switch 706 is shifted to the terminal A in this state, as explained above, the image data obtained in the A/D converter 705 are supplied from the switch 706 to a camera signal processing circuit 707.
The camera signal processing circuit 707 applies a predetermined signal processing to the image data from the switch 706, and supplies an encoder 708 with thus processed image data.
The encoder 708 applies a modulation according to a predetermined television format to the image data supplied from the camera signal processing circuit 707, and supplies a digital/analog (D/A) converter 709 with thus modulated image data.
The D/A converter 709 converts the image data from the encoder 708 into an analog television signal (image signal) for output from an output terminal Iout.
FIG. 2 shows the arrangement of image taking pixels (hereinafter simply called pixels) on the image-pickup surface of the CCD 703, a distance 800x of the sampling points in the X-direction and a distance 800y of the sampling points in the Y-direction.
As shown in FIG. 2, the sampling point for a given pixel 703n is defined in the X- and Y-directions by 800xn and 800yn (indicated by arrows) and can be regarded as a single point for each pixel.
Since such arrangement of the pixels in the X- and Y-directions determines the upper limit of the spatial frequency of the image that can be picked up, the frequency characteristics of the above-mentioned optical low-pass filter 503 are determined according to the range of the spatial frequency of the picked-up image mentioned above.
Consequently the video camera 700 is provided with the optical low-pass filter 503 coaxially with the unrepresented plural optical lenses and is so designed as to cut off, by such optical low-pass filter 503, the high spatial frequencies which generate fold-back noises at the photoelectric conversion by the CCD 703.
Also the video camera 700 realizes the image enlarging function solely by the focal length determined by the zoom lens 701.
More specifically, the image angle of the analog television signal released by the D/A converter 709 is same as the image angle (hereinafter also called image size) of the image focused on the image-pickup surface of the CCD 703.
When an image magnification rate is designated by the manipulation of a zoom switch 712, the control circuit 714 supplies a driver 715 with a drive command based on the manipulation of the zoom switch 712.
The driver 715 drives an actuator 716 based on the drive command from the control circuit 714.
According to the driving of the actuator 716, the zoom lens 701 displaces parallel to the optical axis.
In this manner the above-mentioned focal length is determined, whereby obtained is an image of the magnification rate designated by the manipulation of the zoom switch 712.
In the following there will be explained the operation of the video camera 700, which is performed when the electronic teleconverter function is turned on by placing the electronic teleconverter switch 710 at the terminal B.
As in the above-explained case where the electronic teleconverter function is turned off, the image of an unrepresented object is transmitted through unrepresented plural optical lenses, the zoom lens 701, the optical low-pass filter 702, the CCD 703, the amplifier circuit 704 and the A/D converter 705 thereby being converted into image data, which are supplied to the terminal A of the switch 706.
The image data released from the A/D converter 705 are also supplied to an electronic teleconverter extracting circuit 715.
The electronic teleconverter extracting circuit 715 applies an image extracting process, to be explained later, to the image data from the A/D converter 705, and supplies the terminal B of the switch 706 with the image data subjected to such extracting process.
If the shutter is turned off in this state by shifting a shutter switch 711 to a terminal A, the switch 706 is also switched to the terminal A under the control of the control circuit 714.
Consequently the image data released from the A/D converter 705 are supplied to the camera signal processing circuit 707 through the switch 706.
In this case, the image data of an image size equal to that of the image focused on the image-pickup surface of the CCD 703 are supplied to the camera signal processing circuit 707, and are output from the output terminal Iout through the encoder 708 and the D/A converter 709.
On the other hand, if the shutter is turned on by switching the shutter switch 711 to the terminal B, the switch 706 is also switched to the terminal B under the control of the control circuit 714.
Consequently the extracted image data output from the electronic teleconverter extracting circuit 715 are supplied to the camera signal processing circuit 707.
Thus, in this case, while the shutter is turned on, the extracted image data obtained in the electronic teleconverter extracting circuit 715 are supplied to the camera signal processing circuit 707, and are output from the output terminal Iout through the encoder 708 and the D/A converter 709.
In the following there will be explained the extracting process of the above-mentioned electronic teleconverter extracting circuit 715.
As an example, in case the CCD 703 picks up image 901d within a frame 901 shown in FIG. 3A, the image data based on such image 901d are supplied to the electronic teleconverter extracting circuit 715.
The electronic teleconverter extracting circuit 715 extracts, from the supplied image data (image 901d), image data (image 902d) within a frame 902 of an area corresponding to xc2xc of that of the frame 901 and enlarges thus extracted image data twice vertically and horizontally to obtain enlarged image data (image 902dxe2x80x2) as shown in FIG. 3B.
Consequently, when the electronic teleconverter function is turned on and the shutter is also turned on, the camera signal processing circuit 707 receives, through the switch 706, such enlarged image data (image 902dxe2x80x2) as the extracted image data or the image data after the enlargement by the electronic teleconverter.
However, in such video camera 700, for obtaining the enlarged image data (image 902dxe2x80x2) as shown in FIG. 3B by the electronic teleconverter extracting circuit 715, there is required a process in which the pixel information, in the vertical direction (X-direction), of the image 902d shown in FIG. 3A, reduced to xc2xd by extraction, is rearranged in the vertical direction with a pitch equal to twice of that of the pixel information prior to extraction, and that the pixel information in the horizontal (Y) direction, reduced to xc2xd by extraction, is rearranged in the horizontal direction with a pitch equal to twice of that of the pixel information prior to extraction.
In this operation, as the gap of the neighboring pixels is doubled in comparison with that prior to enlargement, there is also required an interpolation process of interpolating the gap with the average value of the two neighboring pixel information.
Since the pixel information of the image 902d prior to the enlargement by the electronic teleconverter function is used to generate the image 902dxe2x80x2 of a size which basically requires a quadrupled amount of pixel information, the image 902dxe2x80x2 becomes inevitably coarse in resolution, because of the limited amount of pixel information, even despite of the interpolating operation mentioned above.
In consideration of the foregoing, a first object of the present invention is to provide an image processing apparatus and an image pickup apparatus, capable of an enlarged image of high image quality, even in case a part of the image is extracted and enlarged.
A second object of the present invention is to improve the image quality at the image enlargement, by utilizing the pixel displacement technology.
The above-mentioned objects can be attained, according to a preferred embodiment of the present invention, by an image processing apparatus comprising photoelectric conversion means for converting an optical object image, focused on an image-pickup surface thereof, into an electrical image signal; a relative position displacement means for displacing the relative position of the object image focused on the image-pickup surface and the image-pickup surface by a predetermined amount; plural memory means for storing the image signal obtained by the photoelectric conversion means at each displacement executed by the relative position displacement means; a synthesis means for synthesizing the image signals stored in the plural memory means; extraction means for extracting a partial image from the synthesized image signal obtained by the synthesis means, thereby generating an extracted image signal; and enlargement means for enlarging the extracted image signal, generated in the extraction means, thereby generating an enlarged image signal.
A third object of the present invention is to provide an image processing apparatus capable of constantly providing highest image quality both in a moving image processing mode and in a still image processing mode.
The above-mentioned object can be attained, according to a preferred embodiment of the present invention, by an image processing apparatus capable of selectively setting a moving image processing mode and a still image processing mode, comprising photoelectric conversion means for converting an optical object image, focused on an image-pickup surface thereof, into an electrical image signal; a relative position displacement means for displacing the relative position of the object image focused on the image-pickup surface and the image-pickup surface by a predetermined amount; plural memory means for storing the image signal obtained by the photoelectric conversion means at each displacement executed by the relative position displacement means; synthesis means for synthesizing the image signals stored in the plural memory means; extraction means for extracting a partial image from the synthesized image signal obtained by the synthesis means, thereby generating an extracted image signal; and enlargement means for enlarging the extracted image signal, generated in the extraction means, thereby generating an enlarged image signal, wherein the cycle period of displacement in the relative position displacement means is set according to the selected image processing mode.
A fourth object of the present invention is to provide an image pickup apparatus capable of reading plural high-quality images having different aspect ratios of the image area.
A fifth object of the present invention is to provide an image processing apparatus in which the optical low-pass filter is constituted by a pixel displacement system.
A sixth object of the present invention is to provide an image pickup apparatus capable of enlargement, reduction and a change in the aspect ratio of the image without deterioration in the image quality.
A seventh object of the present invention is to optimize a blur correction circuit, an optical low-pass filter etc. according to the selection between a moving image taking mode and a still image taking mode.
Still other objects of the present invention, and the features thereof, will become fully apparent from the following description which is to be taken in conjunction with the attached drawings.